Content compression in networks

Abstract

Described is transparently compressing content for network transmission, including end-to-end compression. An end host or middlebox device sender sends compressed packets to an end host or middlebox device receiver, which decompresses the packets to recover the original packet. The sender constructs compressed packets including references to information maintained at the receiver, which the receiver uses to access the information to recreate actual original packet content. The receiver may include a dictionary corresponding to the sender, e.g., synchronized with the sender's dictionary. Alternatively, in speculative compression, the sender does not maintain a dictionary, and instead sends a fingerprint (hash value) by which the receiver looks up corresponding content in its dictionary; if not found, the receiver requests actual content. Scheduling to maintain fairness and smoothing bursts to coexist with TCP congestion control are also described, as are techniques for routing compressed data over networked end hosts and/or compression-enabled middlebox devices.