Research in Zero-Shot Coordination and Delay Graph Neural Networks

这项研究是在AIMS CDT第一年进行的第二个小型项目的基础上进行的，该项目在图机器学习（ML）领域。图结构数据提供了表格数据之外的额外结构和关系信息，允许对数据域进行几何解释，并原则性地合并归纳偏差（P. W. Battaglia et al. 2018; Bronstein et al. 2021; J. Zhou et al. 2020）。图神经网络中的主导范式，消息传递（Gilmer et al. 2017），在经典情况下只允许局部节点交互，随后会出现过度平滑和过度挤压等问题，从而降低性能（Di Giovanni，Giusti，et al. 2023; Nt and Maehara 2019; Oono and Suzuki 2019; Topping et al. 2021）。方法来解决这些问题并改进经典的消息传递，例如图重新连接（Gasteiger等人，2019年; Topping等人，2021年），多跳消息传递（Abboud et al. 2022; Abu-El-Haija，Perozzi，et al. 2019）和图变换器（Dwivedi and Bresson 2020; Rampasek et al. 2022; Vaswani et al. 2017），但它们通常会淡化或丢弃拓扑提供的归纳偏差，而不是将其纳入消息传递过程。为了更好地利用这种归纳偏差，我们可能不仅要用它来确定图中的两个节点是否以及如何交互，还要确定何时交互。到目前为止，这项研究已经产生了一份会议论文DRew（Gutteridge et al. 2023），该论文已在ICML 2023上被接受。这是第一个工作，考虑这种自适应信息流的图形神经网络，并在正在进行的和未来的项目，我希望继续调查这个框架，静态图和远程交互，但也为应用程序，如时间图，点云和蛋白质design.This建议包括机器学习研究，其中福尔斯属于EPSRC的研究领域的工程和信息技术。目前还没有明确的行业合作，但这项研究在计算化学等领域有潜在的应用。

This research follows on from the second mini-project undertaken in the first year of the AIMS CDT, in the area of graph machine learning (ML).Graph neural networks have enjoyed tremendous popularity in recent years. Graph-structured data provides additional structural and relational information beyond tabular data, allowing for geometric interpretation of data domains and the principled incorporation of inductive biases (P. W. Battaglia et al. 2018; Bronstein et al. 2021; J. Zhou et al. 2020). The dominant paradigm in graph neural networks, message passing (Gilmer et al. 2017), permits only local node interactions in the classical case, and subsequently suffers from issues such as over-smoothing and over-squashing which reduce performance (Di Giovanni, Giusti, et al. 2023; Nt and Maehara 2019; Oono and Suzuki 2019; Topping et al. 2021). Methods to address such issues and improve on classical message passing, such as graph rewiring (Gasteiger et al. 2019; Topping et al. 2021), multi-hop message-passing (Abboud et al. 2022; Abu-El-Haija, Perozzi, et al. 2019) and graph Transformers (Dwivedi and Bresson 2020; Rampasek et al. 2022; Vaswani et al. 2017), but often they dilute or throw away the inductive bias provided by topology, rather than incorporating it into the message passing process. To make better use of this inductive bias, we may want to use it to determine not only whether and how two nodes in a graph interact, but also when. This research has so far resulted in a conference paper, DRew (Gutteridge et al. 2023), which was accepted at ICML 2023. It is the first work to consider such adaptive information flow in graph neural networks, and in ongoing and future projects I hope to continue to investigate this framework, for static graphs and long-range interactions, but also for applications such as temporal graphs, point clouds and protein design.This proposal consists of machine learning research, which falls under the EPSRC research areas of engineering and information technologies. There is no explicit industry collaboration, but the research has potential applications in areas such as computational chemistry.