面向多数据中心的LHAASO科学大数据管理系统及关键技术研究

LHAASO实验数据具有数据量大、结构复杂、实时性强、多中心处理（稻城、北京、成都）、国际共享等特点，是我国目前产生数据量最多的大科学装置之一，现有的数据管理系统面临着前所未有的挑战。比如，站点存储系统与网格存储系统相互隔离，小型站点无法加入，难以实现资源的全面共享；数据管理系统缺乏与科学数据分析的自动协同，效率低且易于出错；在远程站点进行数据分析须提前订阅和传输数据，不能满足云计算时代按需访问的需求；数据在多中心站点及合作组站点之间无法自动分布和迁移等。本项目提出一种面向多数据中心的科学数据管理架构，采用策略存储驱动的方式实现分层次多站点的科学数据管理，基于深度学习方法实现智能化的数据存储，结合流数据缓存技术实现远程站点按需访问及主动的数据服务。本项目将大幅提高LHAASO科学数据管理系统的功能、性能和可用性，为LHAASO产生高质量的物理成果奠定坚实的基础，具有良好的应用推广价值。

LHAASO experimental data has the characteristics of large amount of data, complex structure, strong real-time, multi-center processing (Daocheng, Beijing, Chengdu), international sharing and so on. It is one of the largest scientific devices in China, and the existing data management system is facing unprecedented challenges. For example, the site storage system and grid storage system are isolated from each other, and small-scale sites cannot join in, so it is difficult to realize the full sharing of resources; The data management system lacks automatic coordination with scientific data analysis, is inefficient and prone to errors. Data analysis at remote sites must be Subscription and transmission of data in advance cannot meet the needs of on-demand access in the era of cloud computing; data cannot be automatically distributed and migrated between multi-center sites and cooperative site sites. This project proposes a multi-data center-oriented scientific data management architecture，which adopts a strategy storage-driven approach to realize hierarchical and multi-site scientific data management, realizes intelligent data storage and management based on a deep learning method, and realizes remote site on-demand access and active data service in combination with streaming data caching technology. This project will greatly improve the function, performance and usability of LHAASO scientific data management system, laying a solid foundation for LHAASO to produce high-quality physical results. At the same time, it can also be applied to JUNO, HEPS and other major scientific and technological infrastructure in other countries, with good application and promotion value.