Deep Learning for Astronomically Big Data

SKA图像形成的软实时限制意味着负责创建图像数据产品的科学数据处理器（SDP）将需要约0.5ExaFlops的处理能力，每天产生大约1PB的数据产品。对于天文学家来说，数据产品将通过光纤网络运送到世界各地的SKA区域数据中心。无线电干涉仪的成像依赖于一种处理模型，该模型将数据集从其原生傅立叶测量基础反转成图像。对于SKA来说，这些单独的图像立方体非常大（平均0.25PB），每个图像立方体将包含数万到数十万个不同的天文来源。为了科学开发，需要自动识别这些数据中的对象并对其进行分类。这种操作的机器学习方法已经开始在天体物理学的各个领域得到考虑。对于具有一系列测量和编目特征的先前识别对象的分类，随机森林分类非常流行；然而，在射电天文学中，卷积神经网络的使用已经开始成为一种潜在的机制，可以在识别的同时直接在图像数据中对物体进行分类。这类应用在天体物理学中仍处于起步阶段，考虑如何将这些方法应用于与SKA同等体积的数据集尚不清楚。缩放机器学习方法来处理SKA大小的图像立方体将是世界各地SKA区域中心面临的关键大数据挑战。此外，区域中心在高级图象分析方面还有进一步的潜在优势。不受SKA SDP的实时处理约束，将图像形成纳入其处理模型的机器学习方法可以提供显着增强的输出。傅里叶图像的形成本质上加强了输出图像的特征，这是由于（例如）应用的傅里叶分量权重会使仅基于输出图像产品的分类产生偏差。将傅里叶数据直接结合到深度学习方法中可以提供额外的信息，从而改进分类。在这个项目中，来自LOFAR望远镜的数据将被用作最接近SKA的可用模拟数据。该项目将使用作为LOFAR磁性关键科学项目（MKSP）的一部分获得的GOODS-N油田的深层数据。这些数据是一个深度调查领域，将提供大量的数据在同一区域的天空。

The soft-real time constraints on SKA image formation mean that the science data processor (SDP) responsible for creating image data products will require a processing power of ~0.5ExaFlops, producing approximately 1PB of data products per day. For astronomers, the data products will then be shipped around the world over a fibre network to the SKA regional data centres. Imaging for radio interferometers relies on a processing model that inverts datasets from their native Fourier measurement basis to form an image. For the SKA these individual image cubes are very large (0.25PB on average) and will each contain tens to hundreds of thousands of different astronomical sources. For scientific exploitation, it will be necessary to automatically identify the objects in these data and classify them. Machine learning approaches for such an operation have started to be considered in astrophysics across a range of fields. For classification of previously identified objects with a range of measured and catalogued features, random forest classification is very popular; however in radio astronomy the use of convolutional neural networks has started to emerge as a potential mechanism for classifying objects directly within the image data in parallel with identification. Applications of this sort are still in their infancy in astrophysics and a consideration of how these methods will be applied to datasets with volumes equivalent those from the SKA is unclear. Scaling machine learning approaches to deal with SKA size image cubes will be a key big data challenge for SKA regional centres around the world. In addition, the regional centres have a further potential advantage for advanced image analysis. Not bound by the real-time processing constraints of the SKA SDP, machine learning approaches that incorporate image formation within their processing model could provide significantly enhanced outputs. Fourier image formation by its nature enforces characteristics in the output image due to (e.g.) applied Fourier component weighting that will bias classification based on output image products alone. Incorporating the Fourier data directly into a deep learning approach would provide additional information that could improve classification. For this project, data from the LOFAR telescope will be used as the closest available analogue to that from the SKA. The project will use deep data from the GOODS-N field obtained as part of the LOFAR Magnetism Key Science Project (MKSP). These data are a deep survey field and will provide a large volume of data on the same region of sky.