Algorithms to Improve the Efficiency of Data Compression and Caching on Wide-Area Networks

本研究计划的目标是开发新的无损文本压缩算法和软件工具，以将压缩纳入MIME/HTML标准。该方法包括通过使用字典对文本进行编码以利用语言的自然冗余，然后使用预先存在的压缩算法对其进行压缩。编码方案取决于压缩算法的具体特性，并且基于Burrows-Wheeler变换的Bzip 2算法被用作主要后端压缩算法。在这个研究项目中，我们对编码方案和压缩算法之间的相互作用有了基本的了解。考虑到压缩和通信指标的算法的性能进行测量。基础设施工具是使用字典的动态缓存来开发的，以将压缩嵌入MIME/HTML标准。这项研究将对信息技术的未来产生影响，为数据传输系统提供方法，其中通信带宽是一个溢价和档案存储是一个指数昂贵的奋进。预计新的无损文本压缩算法的压缩率将比最知名的现有压缩算法提高5%至10%，这可能会减少互联网上超过50%的文本流量。实验研究通过报告，会议和期刊论文，博士论文和硕士论文快速传播结果，并将研究知识转移到研究生课程中，从而与教育目标相联系。在此资助下开发的软件工具将通过网站共享。

The goal of this research project is to develop new lossless text compression algorithms and software tools to incorporate compression in MIME/HTML standards. The approach consists of encoding the text to exploit the natural redundancy of a language via the use of a dictionary and then compressing it using a pre-existing compression algorithm. The encoding scheme depends on the specific characteristics of the compression algorithm and the Bzip2 algorithm based on Burrows-Wheeler transform is used as the main backend compression algorithm. A basic understanding of the interaction of the encoding schemes and the compression algorithms is developed in this research project. The performance of the algorithms is measured taking into account both compression and communication metrics. Infrastructure tools are developed using dynamic caching of dictionaries to embed compression into MIME/HTML standards. This research will have impact on the future of information technology by providing methods for data delivery systems where communication bandwidth is at a premium and archival storage is an exponentially costly endeavor. It is expected that the new lossless text compression algorithms will have 5 to 10% improved compression ratio over the best known pre-existing compression algorithms which might translate into a reduction of more than 50% of the text traffic on the Internet. The experimental research is linked to educational goals via rapid dissemination of results via reports, conference and journal papers, doctoral dissertations and masters theses, and transferring the research knowledge into the graduate curriculum. Software tools developed under this grant will be shared via a web site.