Geometric deep learning

我们日常生活中遇到的结构，如社交网络、蛋白质和分子、在线销售的产品等，由于数据点之间的独特关系，如Twitter上两个用户之间的转发次数或两个原子之间的键的类型，最好用图表来解释。与为欧几里得几何设计的经典深度学习相反，图形机器学习（几何深度学习的一个子集）是机器学习的一个领域，它利用额外的非欧几里得结构来改进模型，应用于药物发现、社会学等领域。该项目旨在通过微分几何、光滑形状和流形的研究来增强图机器学习方法。虽然图机器学习的早期工作在很大程度上是从现有深度学习模型（如卷积神经网络）的推广中获得灵感，但微分几何由于其巨大的、在这种背景下很大程度上未被探索的、丰富的概念和多年来建立的理论结果，提供了一个日益复杂的正交来源。该项目的第一项工作“通过曲率理解图上的过压和瓶颈”使用了基于里奇曲率（微分几何中的一个关键概念）的图的新理论结果，设计和评估了一种新的图预处理方法，该方法可以提高现有图神经网络在所有测试数据集上的性能。未来的工作将包括设计新的模型架构，使用曲率来进化信息如何在模型处理特征的同时在图中传播（类似于Beltrami流的基于曲率的神经网络），并扩展现有的基于微分方程的图模型，使用随机微分方程，允许对随时间随机进化的图数据集进行建模。所有这些研究方向都是史无前例的，它们构成了使用不同几何来扩展机器学习应用的令人兴奋的复兴的一部分。该项目属于ESPRC数学科学主题下的几何和拓扑研究领域，因为该研究的新颖性在微分几何和几何深度学习方面具有坚实的基础。它也可能与其他主题相交，因为我们的目标是直接将几何中的强大概念应用于机器学习的世界，从而在生物学和社会学等领域提出新的，有效的方法和模型。

Structures encountered in our everyday lives such as social networks, proteins and molecules, products sold online, and others are best interpreted as a graph due to the unique relationships between datapoints, such as the number of retweets between two users on Twitter or the type of bond between two atoms. As opposed to classical deep learning which was designed for Euclidean, more 'grid-like' geometry, graph machine learning (a subset of geometric deep learning) is a field of machine learning that takes advantage of additional non-Euclidean structure to improve models with applications in drug discovery, sociology and more.This project aims to enhance graph machine learning methods with ideas from differential geometry, the study of smooth shapes and manifolds. While early works in graph machine learning largely take inspiration from generalising existing deep learning models such as convolutional neural networks, differential geometry offers an orthogonal source of increasing sophistication due to its vast, and in this context largely unexplored, wealth of concepts and theoretical results established over many years. The first work in this project "Understanding over-squashing and bottlenecks on graph via curvature" uses new theoretical results on graphs based on Ricci curvature, a key concept from differential geometry, to design and evaluate a new pre-processing method on graphs that improved performance of existing graph neural networks on all datasets tested. Future works will include designing new model architectures using curvature to evolve how information travels across a graph alongside the model processes the features (a curvature-based neural network resembling a Beltrami flow), and extending existing differential equation-based graph models to use stochastic differential equations, allowing for the modelling of graph datasets that evolve randomly through time. All these directions of research are first-of-their-kind and form part of an exciting renaissance of using different geometries to expand the applications in which machine learning can be used.This project falls within the ESPRC Geometry and Topology research area under the Mathematical Sciences theme, as the novelty of the research has a solid footing in differential geometry and geometric deep learning. It also is likely to intersect with other themes, as we aim to directly apply powerful concepts from geometry to the world of machine learning to propose new, efficient methods and models in fields such as biology and sociology.