The Design, Implementation, and Deployment of a System to Transparently Compress Hundreds of Petabytes of Image Files for a File-Storage Service
The Design, Implementation, and Deployment of a System to Transparently Compress Hundreds of Petabytes of Image Files for a File-Storage Service
复制标题
为文件存储服务透明压缩数百 PB 图像文件的系统的设计、实现和部署
DOI：
--
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
Keith Winstein
中科院分区：
文献类型：
--
作者：
D. Horn;Ken Elkabany;Chris Lesniewski;Keith Winstein
我们报告了Lepton的设计，实现和部署，这是一种容忍故障的系统，无损地将JPEG图像压缩到其原始大小的77％。 Lepton替换了基线JPEG压缩的最低层 - A Huffman代码，而并行化算术代码，因此可以快速恢复原始JPEG文件的确切字节。 Lepton匹配最佳先前工作的压缩效率，同时以流式传输方式解码超过九次以上。 Lepton已作为开源软件发布，并已在Dropbox文件存储后端部署了一年。截至2017年2月，它已压缩了超过203个PIB的用户JPEG文件，节省了46多个PIB。
We report the design, implementation, and deployment of Lepton, a fault-tolerant system that losslessly compresses JPEG images to 77% of their original size on average. Lepton replaces the lowest layer of baseline JPEG compression-a Huffman code-with a parallelized arithmetic code, so that the exact bytes of the original JPEG file can be recovered quickly. Lepton matches the compression efficiency of the best prior work, while decoding more than nine times faster and in a streaming manner. Lepton has been released as open-source software and has been deployed for a year on the Dropbox file-storage backend. As of February 2017, it had compressed more than 203 PiB of user JPEG files, saving more than 46 PiB.