Cambridge Astronomical Survey Unit (CASU): Filling the Astronomical Data Lake (2020-2024)

观测天文学正在为发现提供动力，英国在最先进的观测设施方面的投资在资金和智力上都处于领先地位，并从中受益。在过去的十年中，使用WFCAM和Vista对北半球和南半球的近红外进行了全面的成像，英国在这些调查倡议中处于领先地位。这些都提供了广泛的天体物理学领域的关键见解。在接下来的十年里，将迎来大规模的光谱调查，以探测成像揭示的关键人群(无论是星系、恒星还是小行星)。通过参与和定义将在这些设施上进行的具体调查，如Weave、4MOST和MONS，英国已经做好了引领发现的准备。2020年的S有可能成为“全”观测天文学的十年，通过综合成像和光谱尺度数据，能够在所有尺度上深入了解天体物理学。要确保这些设施和调查的科学潜力，就需要有专门知识和系统，以便以最佳方式从数据中提取信息。在这方面，英国凭借剑桥天文勘测单位(CASU)及其被证明有能力向英国和更广泛的社区提供具有成本效益的数据系统而遥遥领先。CASU一直处于测量天文学的前沿，既有从测量数据中最佳地提取知识的开创性技术，也在利用这些信息产生世界领先的研究方面发挥了积极作用。数据处理和科学提供之间的这种协同和反馈被反复证明是确保提供尽可能最佳的科学数据产品以供最广泛的英国和欧洲天文学家使用的关键。在过去的十年中，CASU从Vista、WFCAM和VST成像生成的科学数据产品以及GAIA-ESO VLT光谱学支持了几乎所有英国天体物理研究所的世界级研究计划。CASU正在填满天文数据湖，这是社区能够开采的重要数据资源，与其他多波长数据(如欧几里得、柏拉图)相结合，发现稀有和独特的物体，供ELT或JWST等设施进一步详细研究。CASU的作用在更广泛的背景下得到了认可，ESO依靠CASU提供其目前在ESO调查设施上运行的广泛公共调查的科学数据产品。CASU的设计理念是通过访问PB级数据存储系统和专家管道处理系统，允许演变出针对这种数据雪崩的最佳人体工学解决方案。CASU处理和分析管道的持续开发、维护和运营将确保英国社区处于有利地位，能够迅速科学地利用这些关键的新调查设施的数据。此外，CASU的专业知识对于应对在这些数据中应用机器学习辅助发现所固有的挑战至关重要，并为英国发展UK-LSST伙伴关系提供了潜在的资源。这项赠款建议建立在CASU已经取得的巨大进展的基础上，并要求为2020-2024年期间的以下活动提供资金：-设计和开发下一代ESO/ING广域大规模多路复用光学光谱仪编织、MONS和4MOST的数据管理和分析系统；-为英国领导的Vista大型调查进一步开发、维护、运营和用户支持科学和分析管道；-业务支持、管道处理和为英国领导的ESO VST公共调查进一步开发科学和分析管道；-向CASU科学数据部署高级数据接口，实现数据的机器学习辅助挖掘。

Observational survey astronomy is powering discovery, with the UK leading and benefiting from its investment, both financially and intellectually in state-of-the-art observational facilities. The last decade has seen the delivery of comprehensive imaging of both the northern and southern hemispheres in the near infrared using WFCAM and VISTA, with the UK leading these survey initiatives. These are providing key insights across a wide range of astrophysics. The next decade sees the arrival of large scale spectroscopic surveys to probe key populations (be they galaxies, stars, asteroids) revealed by the imaging. The UK is well poised to lead discovery, through involvement and definition of the specific surveys to be carried out on these facilities such as WEAVE, 4MOST and MOONS. The 2020's has the potential to be the decade of "total" survey astronomy enabling profound insights into astrophysics at all scales through the combination of comprehensive imaging and spectroscopic scale data. Ensuring the scientific potential of these facilities and surveys requires the availability of expertise and systems to optimally extract information from the data. Here the UK has a substantial lead with the Cambridge Astronomical Survey Unit (CASU) and its proven ability to provide cost effective data systems to the UK and wider communities. CASU have been at the forefront of survey astronomy, both pioneering techniques to optimally extract knowledge from survey data, and also in taking a proactive role in exploiting this information to produce world-leading research. This synergy and feedback between data processing and science delivery has been repeatedly demonstrated to be essential in ensuring delivery of the best possible science data products for exploitation by the widest community of UK and European astronomers. In the last decade, CASU generated science data products from VISTA, WFCAM and VST imaging and Gaia-ESO VLT spectroscopy have supported world-class research programmes across almost every UK institute involved in astrophysics. CASU are filling the astronomical data lake, the vital data resource which the community are able to mine, combine with other multi-wavelength data (e.g. Euclid, PLATO) and discover rare and unique objects for further detailed study by facilities such as the ELT or the JWST. The role of CASU has been acknowledged in the wider context, with ESO relying on CASU to provide the science data products from its wide range of public surveys currently running on the ESO survey facilitiesThe CASU design philosophy is to allow the evolution of an optimal ergonomic solution to this avalanche of data, through access to Petabyte scale data storage systems and expert pipeline processing systems. Continued development, maintenance and operation of the CASU processing and analysis pipelines will ensure that the UK community is well positioned to rapidly scientifically exploit the data from these key new survey facilities. In addition CASU expertise will be essential in meeting the challenges inherent in the application of Machine Learning assisted discovery in these data and provide a potential resource for the UK when developing the UK-LSST partnership. This grant proposal builds on the tremendous advances already made by CASU and requests funding for the period 2020-2024 for the following activities:- Design and development of the data management and analysis systems for the next generation of ESO / ING wide field massively multiplexed optical spectrographs WEAVE, MOONS and 4MOST;- Further development, maintenance, operation and user support of the science and analysis pipelines for the UK-led VISTA large scale surveys;- Operational support, pipeline processing and further development of the science and analysis pipelines for the UK-led ESO VST public surveys;- Deployment of advanced data interfaces to the CASU science data, enabling machine learning assisted mining of the data.