Photorealistic Fusion of Virtual and Real Content for Augmented Reality Games

基于计算机图形学的历史和游戏玩家对改进实时渲染的需求，玩家很快就会在增强现实(AR)应用程序中期待传统游戏的照片逼真质量。然而，照片级真实感增强现实的产生受到当前最先进的“逆向建模”中的不足的严重限制。拟议的研究将研究关于场景的哪些信息对于促进插入具有照片真实感质量的虚拟内容是必不可少的。提出将深度估计与逆向渲染相结合，并在输出中加入神经渲染器，可以显著提高实时AR效果的质量。完成后，这项研究有潜力实现超高质量的AR效果，可以产生新的社交游戏体验，覆盖体育赛事的AR效果，并支持AR拱廊和剧院的发展。

Based on the history of computer graphics and the demand gamers have placed on improved real-time rendering, players will soon expect the photorealistic quality of traditional games in Augmented Reality (AR) applications. However, the generation of photorealistic AR is severely limited by deficiencies in the current state-of-the-art in "inverse modelling". The proposed research will study what information about a scene is essential to facilitate the insertion of virtual content with photorealistic quality. It proposes that the quality of real-time AR effects can be drastically improved by combining depth estimation with inverse rendering and including a neural renderer on the output. Upon completion, this research has the potential to enable ultra high quality AR effects, which can produce new social gaming experiences, overlay AR effects on sports events and support the development of AR arcades and theatres.