RI: Small: Inverse Rendering by Co-Evolutionary Learning

这个项目解决了逆向渲染的问题：从单个图像中恢复3D形状，材料和照明。逆绘制是计算机视觉中的一个基本问题；它恢复视觉场景的基本属性，并作为更高级别场景理解的基础，例如识别对象，动作和功能。尽管它具有基本的重要性，但反向渲染仍然很困难。解决逆向渲染可以显著提高计算机视觉，并有利于从自动驾驶到帮助视障人士的各种应用。该项目开发了新的机器学习算法，以推进逆向渲染的艺术状态。此外，该项目还将研究成果纳入各级课程，并招募代表性不足的群体参与这项研究，从而促进教育和多样性。这项研究利用计算机图形学和机器学习推进了逆向渲染技术。特别是，研究团队开发了两个作为对手共同进化的机器学习系统：一个是学习组成3D场景并使用图形引擎渲染图像的渲染系统，另一个是学习从渲染图像中恢复形状、材料和照明的反向渲染系统。为了开发渲染系统，研究小组研究了自适应自动场景构图的新学习算法。为了开发反向渲染系统，研究小组研究了新的学习算法，该算法集成了神经网络和基于物理的视觉。

This project addresses the problem of inverse rendering: recovering 3D shape, material, and lighting from a single image. Inverse rendering is a fundamental problem in computer vision; it recovers the basic properties of a visual scene, and serves as a foundation for higher-level scene understanding such as recognizing objects, actions, and functionalities. Despite its fundamental importance, inverse rendering remains difficult. Solving inverse rendering can significantly advance computer vision and benefit a wide variety of applications from autonomous driving to assisting the visually impaired. This project develops new machine learning algorithms to advance the state of the art of inverse rendering. In addition, the project contributes to education and diversity by integrating research results into courses at various levels and by recruiting underrepresented groups to participate in this research. This research advances inverse rendering technologies using computer graphics and machine learning. In particular, the research team develops two machine learning systems that co-evolve as adversaries: a rendering system that learns to compose 3D scenes and renders images using a graphics engine, and an inverse rendering system that learns to recover shape, material, and lighting from the rendered images. To develop the rendering system, the research team investigates new learning algorithms for adaptive, automatic scene composition. To develop the inverse rendering system, the research team investigates new learning algorithms that integrate neural networks and physics-based vision.