A common real time framework for SuperKEKB and Hyper Suprime-Cam at Subaru telescope
A common real time framework for SuperKEKB and Hyper Suprime-Cam at Subaru telescope
复制标题
Subaru 望远镜 SuperKEKB 和 Hyper Suprime-Cam 的通用实时框架
DOI：
10.1088/1742-6596/219/2/022012
复制
发表时间：
2010
期刊：
影响因子：
--
通讯作者：
S. Mineo
中科院分区：
文献类型：
--
作者：
S. Lee;R. Itoh;N. Katayama;H. Furusawa;H. Aihara;S. Mineo
由于下一代实验数据量大、数据率高，因此对下一代实验数据进行真实的实时分析是一个挑战。SuperKEKB实验是贝儿实验的升级版，需要处理的数据是现有实验的100倍。为了有效地减少数据，HLT农场需要进行离线数据分析。庞大数据的真实的时间处理也是计划中使用斯巴鲁望远镜进行暗能量调查的关键。该调查的主摄像头名为Hyper Suprime-Cam，由100个CCD组成，每个CCD具有800万像素，总数据量预计将与SuperKEKB相当。测量参数的在线调整是通过真实的时间处理来计划的，这在过去是凭经验完成的。我们开始联合开发SuperKEKB和Hyper Suprime-Cam共享的真实的时间框架。在框架设计中广泛采用并行处理技术，以利用大量具有多核CPU的联网PC。并行处理不仅以琐碎的逐事件方式执行，而且在动态放置在分布式计算节点上的软件模块的流水线中执行。框架中的对象数据流通过对象序列化技术和对象持久化技术实现。运行时监控支持直方图和N元组的动态收集。介绍了该框架的详细设计和发展现状。
The real time data analysis at next generation experiments is a challenge because of their enormous data rate and size. The SuperKEKB experiment, the upgraded Belle experiment, requires to process 100 times larger data of current one. The offline-level data analysis is necessary in the HLT farm for the efficient data reduction. The real time processing of huge data is also the key at the planned dark energy survey using the Subaru telescope. The main camera for the survey called Hyper Suprime-Cam consists of 100 CCDs with 8 mega pixels each, and the total data size is expected to become comparable with that of SuperKEKB. The online tuning of measurement parameters is being planned by the real time processing, which was done empirically in the past. We started a joint development of the real time framework to be shared both by SuperKEKB and Hyper Suprime-Cam. The parallel processing technique is widely adopted in the framework design to utilize a huge number of network-connected PCs with multi-core CPUs. The parallel processing is performed not only in the trivial event-by-event manner, but also in the pipeline of the software modules which are dynamically placed over the distributed computing nodes. The object data flow in the framework is realized by the object serializing technique with the object persistency. On-the-fly collection of histograms and N-tuples is supported for the run-time monitoring. The detailed design and the development status of the framework is presented.