GNOMON: Deep Generative Models in non-Euclidean Spaces for Computer Vision & Graphics

GNOMON：计算机视觉非欧几里得空间中的深度生成模型

• 批准号: EP/X011364/1
• 负责人: Stefanos Zafeiriou
• 金额: $ 134.24万
• 依托单位: Imperial College London
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FX011364%2F1
• 关键词: GNOMON; Deep; Generative; Models; non

Over the past decade, deep learning methods have had an enormous impact on the academic and industrial worlds, opening new multi-billion markets ranging from driver-less cars to speech recognition and machine translation. Deep learning has been an emerging technology for decades; it took an orchestrated scientific and engineering effort as well as harnessing of the increasing computational power and large datasets to achieve an overarching technological and societal impact. Most of the successful deep learning methods such as Deep Convolutional Neural Networks (DCNNs) are based on classical signal/image processing models that limit their applicability to data with underlying Euclidean grid-like structure, e.g., 2D/3D images or audio signals. Non-Euclidean (graph-or manifold-structured) data are becoming increasingly abundant; prominent examples include 3D objects (represented as meshes or point clouds) in CV and graphics, as well as social networks, graphs of molecules, and interactomes. Until recently, this has been a significant obstacle precluding the adoption of ML tools in some of the most promising fields. To bridge the gap between Euclidean (e.g., images, videos & speech) and non-Euclidean (e.g., graph and manifolds) ML umbrella terms have recently been coined, such as ''Geometric Deep Learning'' (GDL).Such methods have gained a keen interest in the ML community the past couple of years since graphs can model very abstract systems of relations or interactions, and thus potentially applied across the board. Recent successful examples of the application of non-Euclidean deep learning are as diverse as semantic segmentation on meshes and point clouds, drug-design and event classification in particle physics. Nevertheless, the focus is mainly on discriminative approaches (e.g., classification and segmentation problems) and limited progress has been made towards generative methodologies (i.e., unsupervised methodologies that model the distribution of data) on non-Euclidean spaces. The drawback of discriminative methodologies is that they require a massive amount of labelled, mainly manually, data, which is very expensive, or even impossible to find in many settings. On the other hand, generative approaches can operate in unsupervised scenarios and can even be used to produce data that can be utilised to train discriminative approaches. Currently, available generative frameworks have been developed primarily for Euclidean data (e.g., images, videos) and are not suitable for the non-Euclidean setting.GNoMON aims at bridging this gap by developing a mathematically principled framework for designing and implementing Generative Models for non-Euclidean domains such as graphs or manifolds. We will explore challenging problems in 3D CV and graphics. Nevertheless, the developed techniques will be designed in such a way to be general so that can aid the research in many other fields.

在过去的十年中，深度学习方法对学术界和工业界产生了巨大的影响，开辟了从无人驾驶汽车到语音识别和机器翻译等数十亿美元的新市场。几十年来，深度学习一直是一项新兴技术;它需要精心策划的科学和工程努力，以及利用不断增长的计算能力和大型数据集来实现总体的技术和社会影响。大多数成功的深度学习方法，如深度卷积神经网络（DCNN），都是基于经典的信号/图像处理模型，这些模型限制了它们对具有底层欧几里得网格结构的数据的适用性，例如，2D/3D图像或音频信号。非欧几里德（图形或流形结构）数据变得越来越丰富;突出的例子包括CV和图形中的3D对象（表示为网格或点云），以及社交网络，分子图和相互作用组。直到最近，这一直是阻碍ML工具在一些最有前途的领域中采用的一个重大障碍。为了弥合欧几里德之间的差距（例如，图像、视频和语音）和非欧几里德（例如，图和流形）ML总括术语最近被创造出来，例如“几何深度学习”（GDL）。这些方法在过去几年中引起了ML社区的浓厚兴趣，因为图可以对非常抽象的关系或交互系统进行建模，因此可能会广泛应用。最近成功应用非欧几里德深度学习的例子多种多样，如网格和点云的语义分割，药物设计和粒子物理学中的事件分类。然而，重点主要是歧视性方法（例如，分类和分割问题）并且在生成方法学方面取得了有限的进展（即，对数据分布建模的无监督方法）。区别性方法的缺点是，它们需要大量的标记数据，主要是人工标记的数据，这些数据非常昂贵，甚至在许多情况下不可能找到。另一方面，生成方法可以在无监督的场景中运行，甚至可以用来生成可用于训练判别方法的数据。目前，可用的生成框架主要针对欧几里得数据（例如，GNoMON旨在通过开发一个数学原理框架来弥合这一差距，该框架用于为非欧几里德域（如图或流形）设计和实现生成模型。我们将探索3D CV和图形中具有挑战性的问题。尽管如此，开发的技术将被设计成通用的，以便可以帮助许多其他领域的研究。

项目成果

Neural Shading Fields for Efficient Facial Inverse Rendering

用于高效面部逆渲染的神经着色场

• DOI: 10.1111/cgf.14943
• 发表时间: 2023
• 期刊: Computer Graphics Forum
• 影响因子: 2.5
• 作者: Rainer G

Adaptive Spiral Layers for Efficient 3D Representation Learning on Meshes

• DOI: 10.1109/iccv51070.2023.01344
• 发表时间: 2023-10
• 期刊: 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
• 作者: F. Babiloni;Matteo Maggioni;T. Tanay;Jiankang Deng;A. Leonardis;S. Zafeiriou

Relightify: Relightable 3D Faces from a Single Image via Diffusion Models

• DOI: 10.1109/iccv51070.2023.00809
• 发表时间: 2023-05
• 期刊: 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
• 作者: Foivos Paraperas Papantoniou;Alexandros Lattas;Stylianos Moschoglou;S. Zafeiriou