Multi-Scale Feature Extraction for Spherical CNNs

点击翻译按钮获取中文摘要

The authors of the spherical CNN (PDO-eS2CNN [1]) define a rotation equivariant convolution by mapping spherical signals to the rotational group SO(3). This allows them to define rotationally equivariant convolutions on the sphere. In addition to rotational equivariance, their network also benefits from a kernel with an increased set of six PDOs (partial differential operators), independently parameterising all second order differentials (including cross-terms) and hence attaining a richer set of diffusion characteristics. Importantly, we note that these improvements are entirely orthogonal to the architectural benefits of a Feature Pyramid Network (FPN). Hence, we propose incorporating feature pyramids into their approach would further improve the state-of-the-art performance, demonstrate rotational equivariance, and attain optimal parameter efficiency.State-Of-The-Art Multi-Scale Feature ExtractionA second improvement to spherical CNNs would be to implement the more sophisticated FPN architectures found in state-of-the-art object detection models [2, 3]. These works incorporate repeated blocks of top-down and bottom-up pathways, in what they refer to as multi-scale feature fusion (MSFF). By doing this, they learn improved feature pyramids which are informed by semantic content at all scales. Separately, planar segmentation networks such as [4, 5, 6] achieve improved performance using atrous convolutions. This approach can extract features at multiple scales without needing to exploit the hierarchical structure of an encoder network. We note an analogous prescription can be performed with PDOs on icosahedral meshes by computing diffusion characteristics on a wider neighbourhoods on a central vertex (i.e. 2-ring, 3-ring, etc, see figure 2). The authors of [6] identify the key issues with atrousconvolutions failing to capture long range information due to image boundary effects, and attempt to address this by using techniques such as global average pooling. However, we note that since our domain is inherently edgeless these effects do not apply, and hence atrous convolutions may have some natural synergy with spherical data.Research PlanThe implementation of FPNs into the proposed SO(3)-equivariant network will be particularly challenging and could take substantial implementation time because the original authors are yet to publish their code. However, their highly related planar PDO-based work [7] has been published with code and would illuminate some of the key implementation details. The second avenue of this research concerns generalizing multi-scale feature extraction methods found in the SOTA planar architectures. Finally, we intend to find an optimal combination of these improvements through testing on a wider variety of datasets, e.g. QM7 atomization energy prediction [8], or CAM5 climate segmentation [9] tasks.