Algorithms to Improve the Efficiency of Data Compression and Caching on Wide-Area Networks

点击翻译按钮获取中文摘要

The goal of this research project is to develop new lossless text compression algorithms and software tools to incorporate compression in MIME/HTML standards. The approach consists of encoding the text to exploit the natural redundancy of a language via the use of a dictionary and then compressing it using a pre-existing compression algorithm. The encoding scheme depends on the specific characteristics of the compression algorithm and the Bzip2 algorithm based on Burrows-Wheeler transform is used as the main backend compression algorithm. A basic understanding of the interaction of the encoding schemes and the compression algorithms is developed in this research project. The performance of the algorithms is measured taking into account both compression and communication metrics. Infrastructure tools are developed using dynamic caching of dictionaries to embed compression into MIME/HTML standards. This research will have impact on the future of information technology by providing methods for data delivery systems where communication bandwidth is at a premium and archival storage is an exponentially costly endeavor. It is expected that the new lossless text compression algorithms will have 5 to 10% improved compression ratio over the best known pre-existing compression algorithms which might translate into a reduction of more than 50% of the text traffic on the Internet. The experimental research is linked to educational goals via rapid dissemination of results via reports, conference and journal papers, doctoral dissertations and masters theses, and transferring the research knowledge into the graduate curriculum. Software tools developed under this grant will be shared via a web site.