Scientific Data Lossless Compression Using Fast Neural Network
Scientific Data Lossless Compression Using Fast Neural Network
复制标题
使用快速神经网络进行科学数据无损压缩
DOI：
10.1007/11759966_192
复制
发表时间：
2006
期刊：
影响因子：
--
通讯作者：
Yan Fu
中科院分区：
文献类型：
--
作者：
Junlin Zhou;Yan Fu
科学计算会从复杂的模拟中产生大量数据，通常需要几个结核病，一般压缩方法在这些数据上不能具有良好的性能。神经网络有可能将数据压缩算法扩展到当前正在使用的字符级别（N-Gram模型）之外，但通常是因为它们太慢而无法实用而避免使用。我们使用基于最大熵和算术编码器的快速神经网络提出了一种无损压缩方法，以成功。压缩机是使用2层快速神经网络来预测概率分布的比特预测算术编码器。在训练阶段，改进的自适应可变学习率将优化用于快速收敛培训。所提出的压缩机比流行的压缩机（BZIP，ZZIP，LZO，UCL和DFLATE）在LARED-P数据集上产生的压缩更好，在实际应用的时间和空间上也具有竞争力。
Scientific computing generates huge loads of data from complex simulations, usually takes several TB, general compression methods can not have good performance on these data. Neural networks have the potential to extend data compression algorithms beyond the character level(n-gram model) currently in use, but have usually been avoided because they are too slow to be practical. We present a lossless compression method using fast neural network based on Maximum Entropy and arithmetic coder to succeed in the job. The compressor is a bit-level predictive arithmetic encoder using a 2 layer fast neural network to predict the probability distribution. In the training phase, an improved adaptive variable learning rate is optimized for fast convergence training. The proposed compressor produces better compression than popular compressors(bzip, zzip, lzo, ucl and dflate) on the lared-p data set, also is competitive in time and space for practical application.