Improving ALMA’s data processing efficiency using a holistic approach
Improving ALMA’s data processing efficiency using a holistic approach
复制标题
使用整体方法提高 ALMA 的数据处理效率
DOI：
10.1117/12.2559324
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
J. Ueda
中科院分区：
文献类型：
--
作者：
T. Nakos;H. Francke;K. Nakanishi;D. Petry;T. Stanke;C. Ubach;L. Cerrigone;E. Keller;A. Trejo;J. Ueda
阿尔马（阿塔卡马大型毫米/亚毫米阵列）是世界上最大的毫米/亚毫米波段地面观测设施。阿尔马的突出特点之一是致力于向天文学界提供的校准和成像数据产品的质量保证。数据管理小组（DMG）负责阿尔马数据的数据处理、审查和交付，由分布在14个国家的阿尔马区域中心（ARC）和联合阿尔马观测站（JAO）的大约60名数据简化专家组成。由于每年的吞吐量超过3，000个数据集，因此在数据采集后30天内交付可流水线传输的数据产品是一个巨大的挑战。本文介绍了（a）阿尔马数据处理的历史，（B）我们的团队曾经和仍然面临的挑战，（c）为降低运营风险而采取的方法，（d）正在进行的优化举措，（e）当前的数据处理状态，（f）正在采取的战略，以便在从现在开始的几个周期内，一个由大约10名数据简化员组成的小组可以处理和审查在一个观测周期中收集的大约80%的数据集，最后，（g）ARC在处理其余数据集方面的重要作用。
ALMA (Atacama Large Millimeter/submillimeter Array) is the world's largest ground-based facility for observations in the millimeter/submillimeter regime. One of ALMA's outstanding characteristics is the large effort dedicated to the quality assurance (QA) of the calibrated and imaged data products offered to the astronomical community. The Data Management Group (DMG), in charge of the data processing, review, and delivery of the ALMA data, consists of approximately 60 experts in data reduction, from the ALMA Regional Centers (ARCs) and the Joint ALMA Observatory (JAO), distributed in fourteen countries. With a throughput of more than 3,000 datasets per year, meeting the goal of delivering the pipeline-able data products within 30 days after data acquisition is a huge challenge. This paper presents (a) the history of data processing at ALMA, (b) the challenges our team had and is still facing, (c) the methodology followed to mitigate the operational risks, (d) the ongoing optimization initiatives, (e) the current data processing status, (f) the strategy which is being followed so that, in a few Cycles from now, a team of approximately 10 data reducers (DRs) at JAO can process and review some 80% of the datasets collected during an observing cycle, and, finally, (g) the important role of the ARCs for processing the remaining datasets.