Multi-Scale Feature Extraction for Spherical CNNs

球形 CNN 的多尺度特征提取

• 批准号: 2741382
• 金额: --
• 依托单位: University of Edinburgh
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2022
• 资助国家: 英国
• 起止时间: 2022 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2741382
• 关键词: Multi; Scale; Feature; Extraction; Spherical

The authors of the spherical CNN (PDO-eS2CNN [1]) define a rotation equivariant convolution by mapping spherical signals to the rotational group SO(3). This allows them to define rotationally equivariant convolutions on the sphere. In addition to rotational equivariance, their network also benefits from a kernel with an increased set of six PDOs (partial differential operators), independently parameterising all second order differentials (including cross-terms) and hence attaining a richer set of diffusion characteristics. Importantly, we note that these improvements are entirely orthogonal to the architectural benefits of a Feature Pyramid Network (FPN). Hence, we propose incorporating feature pyramids into their approach would further improve the state-of-the-art performance, demonstrate rotational equivariance, and attain optimal parameter efficiency.State-Of-The-Art Multi-Scale Feature ExtractionA second improvement to spherical CNNs would be to implement the more sophisticated FPN architectures found in state-of-the-art object detection models [2, 3]. These works incorporate repeated blocks of top-down and bottom-up pathways, in what they refer to as multi-scale feature fusion (MSFF). By doing this, they learn improved feature pyramids which are informed by semantic content at all scales. Separately, planar segmentation networks such as [4, 5, 6] achieve improved performance using atrous convolutions. This approach can extract features at multiple scales without needing to exploit the hierarchical structure of an encoder network. We note an analogous prescription can be performed with PDOs on icosahedral meshes by computing diffusion characteristics on a wider neighbourhoods on a central vertex (i.e. 2-ring, 3-ring, etc, see figure 2). The authors of [6] identify the key issues with atrousconvolutions failing to capture long range information due to image boundary effects, and attempt to address this by using techniques such as global average pooling. However, we note that since our domain is inherently edgeless these effects do not apply, and hence atrous convolutions may have some natural synergy with spherical data.Research PlanThe implementation of FPNs into the proposed SO(3)-equivariant network will be particularly challenging and could take substantial implementation time because the original authors are yet to publish their code. However, their highly related planar PDO-based work [7] has been published with code and would illuminate some of the key implementation details. The second avenue of this research concerns generalizing multi-scale feature extraction methods found in the SOTA planar architectures. Finally, we intend to find an optimal combination of these improvements through testing on a wider variety of datasets, e.g. QM7 atomization energy prediction [8], or CAM5 climate segmentation [9] tasks.

球形CNN（PDO-eS 2CNN [1]）的作者通过将球形信号映射到旋转群SO（3）来定义旋转等变卷积。这使得他们能够在球面上定义旋转等变卷积。除了旋转等方差之外，他们的网络还受益于一个增加了六个偏微分算子（PDO）的内核，独立地参数化所有二阶微分（包括交叉项），从而获得更丰富的扩散特性。重要的是，我们注意到这些改进与特征金字塔网络（FPN）的架构优势完全正交。因此，我们建议将特征金字塔纳入他们的方法中，这将进一步提高最先进的性能，证明旋转等方差，并获得最佳的参数效率。最先进的多尺度特征提取对球形CNN的第二个改进是实现最先进的目标检测模型中更复杂的FPN架构[2，3]。这些工作包含了自上而下和自下而上途径的重复块，他们称之为多尺度特征融合（MSFF）。通过这样做，他们学习改进的特征金字塔，这些特征金字塔由所有尺度的语义内容提供信息。单独地，平面分割网络（如[4，5，6]）使用atrous卷积实现了改进的性能。这种方法可以在多个尺度上提取特征，而不需要利用编码器网络的分层结构。我们注意到，通过计算中心顶点上更宽邻域的扩散特性（即2-环，3-环等，参见图2），可以在二十面体网格上使用PDO执行类似的处方。[6]的作者确定了由于图像边界效应而导致atrousconvolutions无法捕获长距离信息的关键问题，并试图通过使用全局平均池化等技术来解决这个问题。然而，我们注意到，由于我们的域本质上是无边的，因此这些效果不适用，因此atrous卷积可能与球形数据有一些自然的协同作用。研究计划将FPNs实现到所提出的SO（3）-等变网络中将特别具有挑战性，并且可能需要大量的实现时间，因为原作者尚未发布他们的代码。然而，他们高度相关的基于平面PDO的工作[7]已经发布了代码，并将阐明一些关键的实现细节。本研究的第二条途径涉及推广多尺度特征提取方法中发现的SOTA平面架构。最后，我们打算通过对更广泛的数据集进行测试来找到这些改进的最佳组合，例如QM 7原子化能量预测[8]或CAM 5气候分割[9]任务。