Revised cloud storage structure for light-weight data archiving in LHD
Revised cloud storage structure for light-weight data archiving in LHD
复制标题
DOI：
10.1016/j.fusengdes.2014.03.046
复制
发表时间：
2014-05
影响因子：
1.7
通讯作者：
H. Nakanishi;Ohsuna Masaki;Kojima Mamoru;Imazu Setsuo;Nonomura Miki;Emoto Masahiko;Yamamoto Takashi;Nagayama Yoshio;Ozeki Takahisa;Nakajima Noriyoshi;Ida Katsumi;Kaneko Osamu
中科院分区：
文献类型：
--
作者：
H. Nakanishi;Ohsuna Masaki;Kojima Mamoru;Imazu Setsuo;Nonomura Miki;Emoto Masahiko;Yamamoto Takashi;Nagayama Yoshio;Ozeki Takahisa;Nakajima Noriyoshi;Ida Katsumi;Kaneko Osamu
LHD数据归档系统新选择了GlusterFS分布式文件系统，取代了现有的云存储软件“IznaStor/dSS”。尽管之前的软件提供了许多有利的功能，如热插拔节点插入、数据文件的内部自动复制和所有成员节点之间的对称负载平衡，但它在从存储节点意外故障中恢复方面表现不佳。一旦发生故障，恢复过程通常至少需要几天时间，有时在cpu负载很重的情况下需要一周以上的时间。在某些情况下，他们陷入了所谓的“脑裂”或“健忘症”状态，无法从中恢复。由于恢复时间与故障节点的容量大小密切相关，因此单个HDD管理比大容量HDD阵列更可取。此外，如果采用其他静态数据分布方法，则可以消除数据位置信息的动态互感知。在本研究中，“OpenStack/Swift”和“GlusterFS”候选中间件使用了半年多的真实海量LHD数据进行了测试，最终选择了GlusterFS来替代目前的IznaStor。它实现了非常有限的云存储功能，但采用了类似raid10的简化结构，因此可以提供更轻量级的读/写能力。由于LABCOM数据系统被实现为独立于存储结构，因此很容易将IznaStor插入到新的GlusterFS上。有效的I/O速度也被证实与磁盘硬件原始性能的估计速度处于同一水平。这一成果可为ITER CODAC和远程存档系统的实现提供参考。
The LHD data archiving system has newly selected GlusterFS distributed filesystem for the replacement of the present cloud storage software named “IznaStor/dSS”. Even though the prior software provided many favorable functionalities of hot plug and play node insertion, internal auto-replication of data files, and symmetric load balancing between all member nodes, it revealed a poor feature in recovering from an accidental malfunction of a storage node. Once a failure happened, the recovering process usually took at least several days or sometimes more than a week with a heavy cpu load. In some cases they fell into the so-called “split-brain” or “amnesia” condition, not to get recovered from it. Since the recovery time tightly depends on the capacity size of the fault node, individual HDD management is more desirable than large volumes of HDD arrays. In addition, the dynamic mutual awareness of data location information may be removed if some other static data distribution method can be applied. In this study, the candidate middleware of “OpenStack/Swift” and “GlusterFS” has been tested by using the real mass of LHD data for more than half a year, and finally GlusterFS has been selected to replace the present IznaStor. It has implemented very limited functionalities of cloud storage but a simplified RAID10-like structure, which may consequently provide lighter-weight read/write ability. Since the LABCOM data system is implemented to be independent of the storage structure, it is easy to plug off the IznaStor and on the new GlusterFS. The effective I/O speed is also confirmed to be on the same level as the estimated one from raw performance of disk hardware. This achievement may be informative to implement the ITER CODAC and the remote archiving system.