Cambridge Astronomical Survey Unit (CASU): Filling the Astronomical Data Lake (2020-2024)

观测调查天文学正在为发现提供动力，英国在最先进的观测设施方面的投资在财政和智力上都处于领先地位并从中受益。在过去十年中，使用WFCAM和VISTA对北方和南半球进行了近红外全面成像，英国在这些调查活动中处于领先地位。这些都为广泛的天体物理学提供了关键的见解。在接下来的十年里，大规模的光谱测量将出现，以探测成像所揭示的关键种群（无论是星系、恒星还是小行星）。联合王国已做好充分准备，通过参与和确定将在WEAVE、4 MOST和MOONS等设施上进行的具体调查，领导发现工作。2020年有可能成为“全面”巡天天文学的十年，通过综合成像和光谱尺度数据的结合，可以对所有尺度的天体物理学进行深刻的见解。要确保这些设施和调查的科学潜力，就需要具备专门知识和系统，以便从数据中最佳地提取信息。在这方面，英国凭借剑桥天文测量单位（CASU）及其向英国和更广泛的社区提供成本效益高的数据系统的能力，处于实质性领先地位。CASU一直处于调查天文学的最前沿，既开创了从调查数据中最佳提取知识的技术，又积极主动地利用这些信息进行世界领先的研究。数据处理和科学交付之间的这种协同作用和反馈已多次被证明是必不可少的，以确保提供最好的科学数据产品，供英国和欧洲天文学家最广泛的社区利用。在过去的十年中，CASU从VISTA，WFCAM和VST成像以及Gaia-ESO VLT光谱学中产生的科学数据产品支持了几乎所有参与天体物理学的英国研究所的世界级研究计划。CASU正在填补天文数据湖，这是社区能够挖掘的重要数据资源，联合收割机与其他多波长数据（例如欧几里得，柏拉图）相结合，并发现稀有和独特的物体，供ELT或JWST等设施进一步详细研究。CASU的作用已经在更广泛的背景下得到了认可，ESO依靠CASU提供科学数据产品，这些产品来自ESO调查设施目前运行的广泛的公共调查。CASU的设计理念是通过访问PB级数据存储系统和专家管道处理系统，为这种雪崩式的数据提供最佳的人体工程学解决方案。CASU处理和分析管道的持续开发，维护和运营将确保英国社区能够快速科学地利用这些关键的新调查设施的数据。此外，CASU的专业知识对于应对这些数据中机器学习辅助发现应用所固有的挑战至关重要，并在发展UK-LSST伙伴关系时为英国提供潜在资源。该拨款提案建立在CASU已经取得的巨大进步的基础上，并要求在2020-2024年期间为以下活动提供资金：-为下一代ESO / ING广域大规模多路复用光谱仪WEAVE，MOONS和4 MOST设计和开发数据管理和分析系统;-进一步开发，维护，运营和用户支持英国领导的VISTA大规模调查的科学和分析管道;- 为英国领导的ESO VST公众调查提供运营支持，管道处理和进一步开发科学和分析管道;-为CASU科学数据部署高级数据接口，使机器学习辅助数据挖掘成为可能。

Observational survey astronomy is powering discovery, with the UK leading and benefiting from its investment, both financially and intellectually in state-of-the-art observational facilities. The last decade has seen the delivery of comprehensive imaging of both the northern and southern hemispheres in the near infrared using WFCAM and VISTA, with the UK leading these survey initiatives. These are providing key insights across a wide range of astrophysics. The next decade sees the arrival of large scale spectroscopic surveys to probe key populations (be they galaxies, stars, asteroids) revealed by the imaging. The UK is well poised to lead discovery, through involvement and definition of the specific surveys to be carried out on these facilities such as WEAVE, 4MOST and MOONS. The 2020's has the potential to be the decade of "total" survey astronomy enabling profound insights into astrophysics at all scales through the combination of comprehensive imaging and spectroscopic scale data. Ensuring the scientific potential of these facilities and surveys requires the availability of expertise and systems to optimally extract information from the data. Here the UK has a substantial lead with the Cambridge Astronomical Survey Unit (CASU) and its proven ability to provide cost effective data systems to the UK and wider communities. CASU have been at the forefront of survey astronomy, both pioneering techniques to optimally extract knowledge from survey data, and also in taking a proactive role in exploiting this information to produce world-leading research. This synergy and feedback between data processing and science delivery has been repeatedly demonstrated to be essential in ensuring delivery of the best possible science data products for exploitation by the widest community of UK and European astronomers. In the last decade, CASU generated science data products from VISTA, WFCAM and VST imaging and Gaia-ESO VLT spectroscopy have supported world-class research programmes across almost every UK institute involved in astrophysics. CASU are filling the astronomical data lake, the vital data resource which the community are able to mine, combine with other multi-wavelength data (e.g. Euclid, PLATO) and discover rare and unique objects for further detailed study by facilities such as the ELT or the JWST. The role of CASU has been acknowledged in the wider context, with ESO relying on CASU to provide the science data products from its wide range of public surveys currently running on the ESO survey facilitiesThe CASU design philosophy is to allow the evolution of an optimal ergonomic solution to this avalanche of data, through access to Petabyte scale data storage systems and expert pipeline processing systems. Continued development, maintenance and operation of the CASU processing and analysis pipelines will ensure that the UK community is well positioned to rapidly scientifically exploit the data from these key new survey facilities. In addition CASU expertise will be essential in meeting the challenges inherent in the application of Machine Learning assisted discovery in these data and provide a potential resource for the UK when developing the UK-LSST partnership. This grant proposal builds on the tremendous advances already made by CASU and requests funding for the period 2020-2024 for the following activities:- Design and development of the data management and analysis systems for the next generation of ESO / ING wide field massively multiplexed optical spectrographs WEAVE, MOONS and 4MOST;- Further development, maintenance, operation and user support of the science and analysis pipelines for the UK-led VISTA large scale surveys;- Operational support, pipeline processing and further development of the science and analysis pipelines for the UK-led ESO VST public surveys;- Deployment of advanced data interfaces to the CASU science data, enabling machine learning assisted mining of the data.