NVST DATA ARCHIVING SYSTEM BASED ON FASTBIT NOSQL DATABASE
NVST DATA ARCHIVING SYSTEM BASED ON FASTBIT NOSQL DATABASE
复制标题
DOI：
10.5303/jkas.2014.47.3.115
复制
发表时间：
2014-06
影响因子：
1
通讯作者：
Yingbo Liu;Feng Wang;Kaifan Ji;H. Deng;W. Dai;Bo Liang
中科院分区：
文献类型：
--
作者：
Yingbo Liu;Feng Wang;Kaifan Ji;H. Deng;W. Dai;Bo Liang
新真空太阳望远镜（NVST）是一个1米真空太阳望远镜，旨在观察太阳活动区域的精细结构。NVST的主要任务是高分辨率成像和光谱观测，包括测量太阳磁场。自2012年开始进行常规观测以来，NVST已经收集了超过2000万个FITS文件，每天最多可产生12万个文件的观测记录。面对海量的文件，如何有效地对文件进行归档和检索成为一个迫切需要解决的问题。在这项研究中，我们实现了一个新的数据归档系统的NVST的基础上Fastbit不仅结构化查询语言（NoSQL）数据库。与关系数据库（即，MySQL; My Structured Query Language），Fastbit数据库在索引和查询性能方面表现出独特的优势。在4000万条记录的大规模数据库中，Fastbit数据库的多字段组合查询响应时间提高了约15倍，完全满足NVST的要求。本文的研究为海量天文数据的归档提供了新的思路，对其他天文望远镜数据管理系统的设计也有一定的借鉴意义。
The New Vacuum Solar Telescope (NVST) is a 1-meter vacuum solar telescope that aims to observe the fine structures of active regions on the Sun. The main tasks of the NVST are high resolution imaging and spectral observations, including the measurements of the solar magnetic field. The NVST has been collecting more than 20 million FITS files since it began routine observations in 2012 and produces a maximum observational records of 120 thousand files in a day. Given the large amount of files, the effective archiving and retrieval of files becomes a critical and urgent problem. In this study, we implement a new data archiving system for the NVST based on the Fastbit Not Only Structured Query Language (NoSQL) database. Comparing to the relational database (i.e., MySQL; My Structured Query Language), the Fastbit database manifests distinctive advantages on indexing and querying performance. In a large scale database of 40 million records, the multi-field combined query response time of Fastbit database is about 15 times faster and fully meets the requirements of the NVST. Our study brings a new idea for massive astronomical data archiving and would contribute to the design of data management systems for other astronomical telescopes.