RI: Small: Inverse Rendering by Co-Evolutionary Learning

RI：小：通过共同进化学习进行逆向渲染

• 批准号: 1617767
• 负责人: Jia Deng
• 金额: $ 45.07万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-06-15 至 2018-10-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1617767&HistoricalAwards=false
• 关键词: RI; Small; Inverse; Rendering; Co

This project addresses the problem of inverse rendering: recovering 3D shape, material, and lighting from a single image. Inverse rendering is a fundamental problem in computer vision; it recovers the basic properties of a visual scene, and serves as a foundation for higher-level scene understanding such as recognizing objects, actions, and functionalities. Despite its fundamental importance, inverse rendering remains difficult. Solving inverse rendering can significantly advance computer vision and benefit a wide variety of applications from autonomous driving to assisting the visually impaired. This project develops new machine learning algorithms to advance the state of the art of inverse rendering. In addition, the project contributes to education and diversity by integrating research results into courses at various levels and by recruiting underrepresented groups to participate in this research. This research advances inverse rendering technologies using computer graphics and machine learning. In particular, the research team develops two machine learning systems that co-evolve as adversaries: a rendering system that learns to compose 3D scenes and renders images using a graphics engine, and an inverse rendering system that learns to recover shape, material, and lighting from the rendered images. To develop the rendering system, the research team investigates new learning algorithms for adaptive, automatic scene composition. To develop the inverse rendering system, the research team investigates new learning algorithms that integrate neural networks and physics-based vision.

这个项目解决了逆向渲染的问题：从单个图像中恢复3D形状，材质和照明。逆向渲染是计算机视觉中的一个基本问题;它恢复了视觉场景的基本属性，并作为更高级别场景理解的基础，如识别对象，动作和功能。尽管其基本的重要性，逆渲染仍然是困难的。解决逆向渲染可以显着推进计算机视觉，并使从自动驾驶到辅助视障人士的各种应用受益。该项目开发新的机器学习算法，以推进逆向渲染的艺术水平。此外，该项目还通过将研究成果纳入各级课程和招募代表性不足的群体参与这项研究，为教育和多样性作出贡献。这项研究利用计算机图形学和机器学习推进了逆向绘制技术。特别是，研究团队开发了两个机器学习系统，它们作为对手共同进化：一个渲染系统，它学习组成3D场景并使用图形引擎渲染图像，另一个逆渲染系统，它学习从渲染图像中恢复形状，材料和照明。为了开发渲染系统，研究小组研究了用于自适应自动场景合成的新学习算法。为了开发逆向渲染系统，研究团队研究了集成神经网络和基于物理的视觉的新学习算法。