Debian delta packages

[Florian Hackenberger]

Hi!

A major headache for most Ubuntu users is the size of packages upgrades, 
especially for those on traffic limited (aka most) broadband lines. Users on 
dial-up cannot update their system at all, because most packages are more 
than a few hundred kB in size. There is a (partial) solution to this problem: 
delta packages. Contrary to rpms there is no tool for generating delta 
packages for deb files (AFAIK). What about implementing support for delta 
packages for the Edgy Eft (+1, if it's too late for Edgy) release? It's 
pretty easy as I will show you.

First my testing results:
Tested 126 package upgrades (those packages I had in my local archive cache).
The overall download size of these packages is 121MB
Using xdelta3 (from http://xdelta.org) and the attached scripts for testing, I 
was able to save ~64MB of download volume. Thats over 52%! One extreme 
example: For the kernel update from 2.6.15-26.47 to 2.6.15-27.48 I had to 
download 21.7 MB. When we implement delta debian package support this update 
would be just 1.55 MB, which just 7% or the original size. Impressed? Well 
thank google for their great xdelta3 GPL code!

Problems:
1) gzip: When gzipping the same file twice, the result differs:
$ diff data.tar data.tar.patched
$
$ gzip data.tar data.tar.patched
$ diff data.tar.gz data.tar.patched.gz
Binary files data.tar.gz and data.tar.patched.gz differ
$
I just hope this does not break apt-get checksum verification.

2) xdelta3.h defines XD3_NODECOMPRESSSIZE, which defines the maximum size it 
will accept for files compressed in external formats. Some files like 
data.tar.gz of the linux-image are bigger than this size. Solutions:
* increase the constant
* decompress the files before using xdelta3

If you want to repeat my tests just download and compile xdelta3, place my 
scripts in the xdelta3 directory and do the following:
./find_double_debs.sh
./delta_script.sh

However, note that these scripts are just a quick hack and have at least one 
known bug (it uses only packages with exactly two versions available).

Please tell me what you think!

Regards,
	Florian

-- 
Florian Hackenberger
student @
University of Technology
Graz, Austria
florian at hackenberger.at
www.hackenberger.at
-------------- next part --------------
A non-text attachment was scrubbed...
Name: delta_script.sh
Type: application/x-shellscript
Size: 1295 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20061014/1c900d82/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: find_double_debs.sh
Type: application/x-shellscript
Size: 754 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20061014/1c900d82/attachment-0001.bin