Large-Scale Analysis of the Docker Hub Dataset
Large-Scale Analysis of the Docker Hub Dataset
复制标题
DOI：
10.1109/cluster.2019.8891000
复制
发表时间：
2019-09
期刊：
影响因子：
--
通讯作者：
Nannan Zhao;Vasily Tarasov;Hadeel Albahar;Ali Anwar;Lukas Rupprecht;Dimitrios Skourtis;Amit Warke;Mohamed Mohamed-Mohamed;A. Butt
中科院分区：
文献类型：
--
作者：
Nannan Zhao;Vasily Tarasov;Hadeel Albahar;Ali Anwar;Lukas Rupprecht;Dimitrios Skourtis;Amit Warke;Mohamed Mohamed-Mohamed;A. Butt
Docker Containers已经成为支持现代企业应用程序的重要解决方案，这是因为执行环境具有非常理想的隔离、低开销和高效打包特性。容器是通过Docker注册表在用户之间共享的图像创建的。Docker注册表存储的数据量非常大；例如，流行的公共注册表Docker Hub存储了至少50万个公共图像。在本白皮书中，我们分析了超过167 TB的未压缩Docker Hub映像，使用多个指标对其进行表征，并评估了Docker Hub中文件级重复数据消除的潜力。我们的分析有助于在为一般容器，特别是Docker注册表设计存储时做出有意识的决策。例如，映像中只有3%的文件是唯一的，这意味着文件级重复数据消除具有为注册表节省存储空间的巨大潜力。我们的发现可以激励和帮助改进注册表的数据缩减、缓存和拉取优化的设计。
Docker containers have become a prominent solution for supporting modern enterprise applications due to the highly desirable features of isolation, low overhead, and efficient packaging of the execution environment. Containers are created from images which are shared between users via a Docker registry. The amount of data Docker registries store is massive; for example, Docker Hub, a popular public registry, stores at least half a million public images. In this paper, we analyze over 167 TB of uncompressed Docker Hub images, characterize them using multiple metrics and evaluate the potential of file-level deduplication in Docker Hub. Our analysis helps to make conscious decisions when designing storage for containers in general and Docker registries in particular. For example, only 3% of the files in images are unique, which means file-level deduplication has a great potential to save storage space for the registry. Our findings can motivate and help improve the design of data reduction, caching, and pulling optimizations for registries.