Geometric deep learning

我们日常生活中遇到的结构，如社交网络、蛋白质和分子、在线销售的产品等，由于数据点之间的独特关系，如Twitter上两个用户之间的转发数量或两个原子之间的键类型，最好用图表来解释。与经典的深度学习不同，它是为欧几里得设计的，更像是“网格”几何，图机器学习（几何深度学习的一个子集）是机器学习的一个领域，它利用额外的非欧几里得结构来改进模型，并将其应用于药物发现，社会学等领域。该项目旨在利用微分几何的思想来增强图机器学习方法，对光滑形状和流形的研究。虽然图机器学习的早期工作很大程度上是从推广现有的深度学习模型（如卷积神经网络）中获得灵感，但微分几何提供了一个日益复杂的正交来源，因为它拥有大量的概念和多年来建立的理论结果，在这种背景下，这些概念和理论结果基本上是未被探索的。该项目的第一项工作“通过曲率理解图上的过度挤压和瓶颈”使用了基于Ricci曲率（微分几何的一个关键概念）的图的新理论结果，以设计和评估一种新的图预处理方法，该方法提高了现有图神经网络在所有测试数据集上的性能。未来的工作将包括设计新的模型架构，使用曲率来发展信息如何在图形中传播，以及模型处理特征（基于曲率的神经网络，类似于Beltrami流），并扩展现有的基于微分方程的图形模型，以使用随机微分方程，允许对随时间随机演变的图形数据集进行建模。所有这些研究方向都是开创性的，并且是使用不同几何来扩展机器学习应用的令人兴奋的复兴的一部分。该项目福尔斯属于ESPRC数学科学主题下的几何和拓扑研究领域，因为该研究的新颖性在微分几何和几何深度学习方面有着坚实的基础。它也可能与其他主题交叉，因为我们的目标是将几何学中的强大概念直接应用于机器学习领域，以在生物学和社会学等领域提出新的，有效的方法和模型。

Structures encountered in our everyday lives such as social networks, proteins and molecules, products sold online, and others are best interpreted as a graph due to the unique relationships between datapoints, such as the number of retweets between two users on Twitter or the type of bond between two atoms. As opposed to classical deep learning which was designed for Euclidean, more 'grid-like' geometry, graph machine learning (a subset of geometric deep learning) is a field of machine learning that takes advantage of additional non-Euclidean structure to improve models with applications in drug discovery, sociology and more.This project aims to enhance graph machine learning methods with ideas from differential geometry, the study of smooth shapes and manifolds. While early works in graph machine learning largely take inspiration from generalising existing deep learning models such as convolutional neural networks, differential geometry offers an orthogonal source of increasing sophistication due to its vast, and in this context largely unexplored, wealth of concepts and theoretical results established over many years. The first work in this project "Understanding over-squashing and bottlenecks on graph via curvature" uses new theoretical results on graphs based on Ricci curvature, a key concept from differential geometry, to design and evaluate a new pre-processing method on graphs that improved performance of existing graph neural networks on all datasets tested. Future works will include designing new model architectures using curvature to evolve how information travels across a graph alongside the model processes the features (a curvature-based neural network resembling a Beltrami flow), and extending existing differential equation-based graph models to use stochastic differential equations, allowing for the modelling of graph datasets that evolve randomly through time. All these directions of research are first-of-their-kind and form part of an exciting renaissance of using different geometries to expand the applications in which machine learning can be used.This project falls within the ESPRC Geometry and Topology research area under the Mathematical Sciences theme, as the novelty of the research has a solid footing in differential geometry and geometric deep learning. It also is likely to intersect with other themes, as we aim to directly apply powerful concepts from geometry to the world of machine learning to propose new, efficient methods and models in fields such as biology and sociology.