On-The-Fly inter-proxy data compression for web access

Abstract

There has been a significant growth in the residential apartment market in Sri Lanka during the last decade. The main reasons behind this upsurge include the growth of population, changes in living patterns and the soaring prices of property particularly Obtaining a reasonable speed for web access is a problem in many areas, especially for small organizations such as schools, dial-up, and mobile users, due to the low bandwidth of the available links. One solution, which is supported by HTTP 1.1, is the compression of web pages, but this needs support by both the web server and web client. As most web servers have not enabled such support, this feature is in limited use. An alternative option is to implement compression between two proxy servers located at each end of the bandwidth-limited link. This dissertation describes the implementation of such a system. A compression scheme was implemented which is transparent to both client and server. Data is compressed at the upstream proxy server of the bandwidth-limited link, and de-compressed at the downstream proxy server of the link. Different types of content are identified based on the content-type HTTP header and different compressors are used on each content-type. HTTP headers and text content-types (html, css, txt etc.) are highly compressible. A number of text compression schemes were evaluated, and, gzip was selected as the compressor for such content. A unique feature of our system is the use of a pre-set dictionary for HTTP header compression, which enabled us to get very good compression ratios. Although jpeg images are already in a compressed format, they can generally be further compressed without excessively degrading the perceived image quality. We do so whenever feasible. Persistent connections over the limited bandwidth links were introduced to eliminate the delay caused by TCP connection establishment. The performance of .the system under different workloads was analyzed, which showed that the system provides a significant improvement in response time over a low-speed connection.